Representational Similarity Analysis for Tracking Neural Correlates of Haptic Learning on a Multimodal Device.

A goal of wearable haptic devices has been to enable haptic communication, where individuals learn to map information typically processed visually or aurally to haptic cues via a process of cross-modal associative learning. Neural correlates have been used to evaluate haptic perception and may provide a more objective approach to assess association performance than more commonly used behavioral measures of performance. In this article, we examine Representational Similarity Analysis (RSA) of electroencephalography (EEG) as a framework to evaluate how the neural representation of multifeatured haptic cues changes with association training. We focus on the first phase of cross-modal associative learning, perception of multimodal cues. A participant learned to map phonemes to multimodal haptic cues, and EEG data were acquired before and after training to create neural representational spaces that were compared to theoretical models. Our perceptual model showed better correlations to the neural representational space before training, while the feature-based model showed better correlations with the post-training data. These results suggest that training may lead to a sharpening of the sensory response to haptic cues. Our results show promise that an EEG-RSA approach can capture a shift in the representational space of cues, as a means to track haptic learning.

A neurocomputational analysis of visual bias on bimanual tactile spatial perception during a crossmodal exposure.

Vision and touch both support spatial information processing. These sensory systems also exhibit highly specific interactions in spatial perception, which may reflect multisensory representations that are learned through visuo-tactile (VT) experiences. Recently, Wani and colleagues reported that task-irrelevant visual cues bias tactile perception, in a brightness-dependent manner, on a task requiring participants to detect unimanual and bimanual cues. Importantly, tactile performance remained spatially biased after VT exposure, even when no visual cues were presented. These effects on bimanual touch conceivably reflect cross-modal learning, but the neural substrates that are changed by VT experience are unclear. We previously described a neural network capable of simulating VT spatial interactions. Here, we exploited this model to test different hypotheses regarding potential network-level changes that may underlie the VT learning effects. Simulation results indicated that VT learning effects are inconsistent with plasticity restricted to unisensory visual and tactile hand representations. Similarly, VT learning effects were also inconsistent with changes restricted to the strength of inter-hemispheric inhibitory interactions. Instead, we found that both the hand representations and the inter-hemispheric inhibitory interactions need to be plastic to fully recapitulate VT learning effects. Our results imply that crossmodal learning of bimanual spatial perception involves multiple changes distributed over a VT processing cortical network.

Cortical representations of phantom movements in lower limb amputees.

Understanding how sensorimotor cortex (SMC) organization relates to limb loss has major clinical implications, as cortical activity associated with phantom hand movements has been shown to predict phantom pain reports. Critically, earlier studies have largely focused on upper limb amputees; far less is known regarding SMC activity in lower limb amputees, despite the fact that this population comprises the majority of major limb loss cases. We aimed to characterize BOLD fMRI responses associated with phantom and sound limb movements to test the hypothesis that SMC organization is preserved in individuals with lower limb loss. Individuals with unilateral or bilateral lower limb loss underwent fMRI scans as they performed simple movements of their sound or phantom limbs. We observed that voluntary movements of the sound and phantom ankles were associated with BOLD signal changes in medial and superior portions of the precentral and postcentral gyri. In both hemispheres, contralateral limb movements were associated with greater signal changes compared to ipsilateral limb movements. Hand and mouth movements were associated with distinct activation patterns localized to more lateral SMC regions. We additionally tested whether activations associated with phantom movements related to self-report assessments indexing phantom pain experiences, nonpainful phantom sensations and phantom movement capabilities. We found that responses during phantom ankle movements did not correlate with any of the composite phantom limb indices in our sample. Our collective results reveal that SMC representations of the amputated limb persist and that traditional somatotopic organization is generally preserved in individuals suffering from lower limb loss.

Auditory and tactile frequency representations are co-embedded in modality-defined cortical sensory systems.

Sensory information is represented and elaborated in hierarchical cortical systems that are thought to be dedicated to individual sensory modalities. This traditional view of sensory cortex organization has been challenged by recent evidence of multimodal responses in primary and association sensory areas. Although it is indisputable that sensory areas respond to multiple modalities, it remains unclear whether these multimodal responses reflect selective information processing for particular stimulus features. Here, we used fMRI adaptation to identify brain regions that are sensitive to the temporal frequency information contained in auditory, tactile, and audiotactile stimulus sequences. A number of brain regions distributed over the parietal and temporal lobes exhibited frequency-selective temporal response modulation for both auditory and tactile stimulus events, as indexed by repetition suppression effects. A smaller set of regions responded to crossmodal adaptation sequences in a frequency-dependent manner. Despite an extensive overlap of multimodal frequency-selective responses across the parietal and temporal lobes, representational similarity analysis revealed a cortical "regional landscape" that clearly reflected distinct somatosensory and auditory processing systems that converged on modality-invariant areas. These structured relationships between brain regions were also evident in spontaneous signal fluctuation patterns measured at rest. Our results reveal that multimodal processing in human cortex can be feature-specific and that multimodal frequency representations are embedded in the intrinsically hierarchical organization of cortical sensory systems.

Principles of tactile search over the body.

Although we routinely experience complex tactile patterns over our entire body, how we selectively experience multisite touch over our bodies remains poorly understood. Here, we characterized tactile search behavior over the full body using a tactile analog of the classic visual search task. On each trial, participants judged whether a target stimulus (e.g., 10-Hz vibration) was present or absent anywhere on the body. When present, the target stimulus could occur alone or simultaneously with distractor stimuli (e.g., 30-Hz vibrations) on other body locations. We systematically varied the number and spatial configurations of the distractors as well as the target and distractor frequencies and measured the impact of these factors on tactile search response times. First, we found that response times were faster on target-present trials compared with target-absent trials. Second, response times increased with the number of stimulated sites, suggesting a serial search process. Third, search performance differed depending on stimulus frequencies. This frequency-dependent behavior may be related to perceptual grouping effects based on timing cues. We constructed linear models to explore how the locations of the target and distractor cues influenced tactile search behavior. Our modeling results reveal that, in isolation, cues on the index fingers make relatively greater contributions to search performance compared with stimulation experienced on other body sites. Additionally, costimulation of sites within the same limb or simply on the same body side preferentially influence search behavior. Our collective findings identify some principles of attentional search that are common to vision and touch, but others that highlight key differences that may be unique to body-based spatial perception.NEW & NOTEWORTHY Little is known about how we selectively experience multisite touch patterns over the body. Using a tactile analog of the classic visual target search paradigm, we show that tactile search behavior for flutter cues is generally consistent with a serial search process. Modeling results reveal the preferential contributions of index finger stimulation and two-site stimulus interactions involving ipsilateral patterns and within-limb patterns. Our results offer initial evidence for spatial and temporal principles underlying tactile search behavior over the body.

EPI distortion correction for concurrent human brain stimulation and imaging at 3T.

BACKGROUND: Transcranial magnetic stimulation (TMS) can be paired with functional magnetic resonance imaging (fMRI) in concurrent TMS-fMRI experiments. These multimodal experiments enable causal probing of network architecture in the human brain which can complement alternative network mapping approaches. Critically, merely introducing the TMS coil into the scanner environment can sometimes produce substantial magnetic field inhomogeneities and spatial distortions which limit the utility of concurrent TMS-fMRI. METHOD AND RESULTS: We assessed the efficacy of point spread function corrected echo planar imaging (PSF-EPI) in correcting for the field inhomogeneities associated with a TMS coil at 3 T. In phantom and brain scans, we quantitatively compared the coil-induced distortion artifacts measured in EPI scans with and without PSF correction. We found that the application of PSF corrections to the EPI data significantly improved signal-to-noise and reduced distortions. In phantom scans with the PSF-EPI sequence, we also characterized the temporal profile of dynamic artifacts associated with TMS delivery and found that image quality remained high as long as the TMS pulse preceded the RF excitation pulses by at least 50 ms. Lastly, we validated the PSF-EPI sequence in human brain scans involving TMS and motor behavior as well as resting state fMRI scans. CONCLUSIONS: Our collective results demonstrate the potential benefits of PSF-EPI for concurrent TMS-fMRI when coil-related artifacts are a concern. The ability to collect high quality resting state fMRI data in the same session as the concurrent TMS-fMRI experiment offers a unique opportunity to interrogate network architecture in the human brain.

Reciprocal Interactions Between Audition and Touch in Flutter Frequency Perception.

In both audition and touch, sensory cues comprising repeating events are perceived either as a continuous signal or as a stream of temporally discrete events (flutter), depending on the events' repetition rate. At high repetition rates (>100 Hz), auditory and tactile cues interact reciprocally in pitch processing. The frequency of a cue experienced in one modality systematically biases the perceived frequency of a cue experienced in the other modality. Here, we tested whether audition and touch also interact in the processing of low-frequency stimulation. We also tested whether multisensory interactions occurred if the stimulation in one modality comprised click trains and the stimulation in the other modality comprised amplitude-modulated signals. We found that auditory cues bias touch and tactile cues bias audition on a flutter discrimination task. Even though participants were instructed to attend to a single sensory modality and ignore the other cue, the flutter rate in the attended modality is perceived to be similar to that of the distractor modality. Moreover, we observed similar interaction patterns regardless of stimulus type and whether the same stimulus types were experienced by both senses. Combined with earlier studies, our results suggest that the nervous system extracts and combines temporal rate information from multisensory environmental signals, regardless of stimulus type, in both the low- and high temporal frequency domains. This function likely reflects the importance of temporal frequency as a fundamental feature of our multisensory experience.

Somatosensory interactions reveal feature-dependent computations.

Our ability to perceive and discriminate textures is based on the processing of high-frequency vibrations generated on the fingertip as it scans across a surface. Although much is known about the processing of vibration amplitude and frequency information when cutaneous stimulation is experienced at a single location on the body, how these stimulus features are processed when touch occurs at multiple locations is poorly understood. We evaluated participants' ability to discriminate tactile cues (100-300 Hz) on one hand while they ignored distractor cues experienced on their other hand. We manipulated the relative positions of the hands to characterize how limb position influenced cutaneous touch interactions. In separate experiments, participants judged either the frequency or intensity of mechanical vibrations. We found that vibrations experienced on one hand always systematically modulated the perception of vibrations on the other hand. Notably, bimanual interaction patterns and their sensitivity to hand locations differed according to stimulus feature. Somatosensory interactions in intensity perception were only marked by attenuation that was invariant to hand position manipulations. In contrast, interactions in frequency perception consisted of both bias and sensitivity changes that were more pronounced when the hands were held in close proximity. We implemented models to infer the neural computations that mediate somatosensory interactions in the intensity and frequency dimensions. Our findings reveal obligatory and feature-dependent somatosensory interactions that may be supported by both feature-specific and feature-general operations. NEW & NOTEWORTHY Little is known about the neural computations mediating feature-specific sensory interactions between the hands. We show that vibrations experienced on one hand systematically modulate the perception of vibrations felt on the other hand. Critically, interaction patterns and their dependence on the relative positions of the hands differed depending on whether participants judged vibration intensity or frequency. These results, which we recapitulate with models, imply that somatosensory interactions are mediated by feature-dependent neural computations.

Multisensory perceptual interactions between higher-order temporal frequency signals.

Naturally occurring signals in audition and touch can be complex and marked by temporal variations in frequency and amplitude. Auditory frequency sweep processing has been studied extensively; however, much less is known about sweep processing in touch because studies have primarily focused on the perception of simple sinusoidal vibrations. Given the extensive interactions between audition and touch in the frequency processing of pure tone signals, we reasoned that these senses might also interact in the processing of higher-order frequency representations like sweeps. In a series of psychophysical experiments, we characterized the influence of auditory distractors on the ability of participants to discriminate tactile frequency sweeps. Auditory frequency sweeps systematically biased the tactile perception of sweep direction. Importantly, auditory cues exerted little influence on tactile sweep direction perception when the sounds and vibrations occupied different absolute frequency ranges or when the sounds consisted of intensity sweeps. Thus, audition and touch interact in frequency sweep perception in a frequency- and feature-specific manner. Our results demonstrate that audio-tactile interactions are not constrained to the processing of simple sinusoids. Because higher-order frequency representations may be synthesized from simpler representations, our findings imply that multisensory interactions in the temporal frequency domain span multiple hierarchical levels in sensory processing. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Touch engages visual spatial contextual processing.

The spatial context in which we view a visual stimulus strongly determines how we perceive the stimulus. In the visual tilt illusion, the perceived orientation of a visual grating is affected by the orientation signals in its surrounding context. Conceivably, the spatial context in which a visual grating is perceived can be defined by interactive multisensory information rather than visual signals alone. Here, we tested the hypothesis that tactile signals engage the neural mechanisms supporting visual contextual modulation. Because tactile signals also convey orientation information and touch can selectively interact with visual orientation perception, we predicted that tactile signals would modulate the visual tilt illusion. We applied a bias-free method to measure the tilt illusion while testing visual-only, tactile-only or visuo-tactile contextual surrounds. We found that a tactile context can influence visual tilt perception. Moreover, combining visual and tactile orientation information in the surround results in a larger tilt illusion relative to the illusion achieved with the visual-only surround. These results demonstrate that the visual tilt illusion is subject to multisensory influences and imply that non-visual signals access the neural circuits whose computations underlie the contextual modulation of vision.

Selective Attention Gates the Interactive Crossmodal Coupling between Perceptual Systems.

Sensory cortical systems often activate in parallel, even when stimulation is experienced through a single sensory modality [1-3]. Co-activations may reflect the interactive coupling between information-linked cortical systems or merely parallel but independent sensory processing. We report causal evidence consistent with the hypothesis that human somatosensory cortex (S1), which co-activates with auditory cortex during the processing of vibrations and textures [4-9], interactively couples to cortical systems that support auditory perception. In a series of behavioral experiments, we used transcranial magnetic stimulation (TMS) to probe interactions between the somatosensory and auditory perceptual systems as we manipulated attention state. Acute TMS over S1 impairs auditory frequency perception when subjects simultaneously attend to auditory and tactile frequency, but not when attention is directed to audition alone. Auditory frequency perception is unaffected by TMS over visual cortex, thus confirming the privileged interactions between the somatosensory and auditory systems in temporal frequency processing [10-13]. Our results provide a key demonstration that selective attention can modulate the functional properties of cortical systems thought to support specific sensory modalities. The gating of crossmodal coupling by selective attention may critically support multisensory interactions and feature-specific perception.

Auditory Frequency Representations in Human Somatosensory Cortex.

Recent studies have challenged the traditional notion of modality-dedicated cortical systems by showing that audition and touch evoke responses in the same sensory brain regions. While much of this work has focused on somatosensory responses in auditory regions, fewer studies have investigated sound responses and representations in somatosensory regions. In this functional magnetic resonance imaging (fMRI) study, we measured BOLD signal changes in participants performing an auditory frequency discrimination task and characterized activation patterns related to stimulus frequency using both univariate and multivariate analysis approaches. Outside of bilateral temporal lobe regions, we observed robust and frequency-specific responses to auditory stimulation in classically defined somatosensory areas. Moreover, using representational similarity analysis to define the relationships between multi-voxel activation patterns for all sound pairs, we found clear similarity patterns for auditory responses in the parietal lobe that correlated significantly with perceptual similarity judgments. Our results demonstrate that auditory frequency representations can be distributed over brain regions traditionally considered to be dedicated to somatosensation. The broad distribution of auditory and tactile responses over parietal and temporal regions reveals a number of candidate brain areas that could support general temporal frequency processing and mediate the extensive and robust perceptual interactions between audition and touch.

Auditory adaptation improves tactile frequency perception.

Our ability to process temporal frequency information by touch underlies our capacity to perceive and discriminate surface textures. Auditory signals, which also provide extensive temporal frequency information, can systematically alter the perception of vibrations on the hand. How auditory signals shape tactile processing is unclear; perceptual interactions between contemporaneous sounds and vibrations are consistent with multiple neural mechanisms. Here we used a crossmodal adaptation paradigm, which separated auditory and tactile stimulation in time, to test the hypothesis that tactile frequency perception depends on neural circuits that also process auditory frequency. We reasoned that auditory adaptation effects would transfer to touch only if signals from both senses converge on common representations. We found that auditory adaptation can improve tactile frequency discrimination thresholds. This occurred only when adaptor and test frequencies overlapped. In contrast, auditory adaptation did not influence tactile intensity judgments. Thus auditory adaptation enhances touch in a frequency- and feature-specific manner. A simple network model in which tactile frequency information is decoded from sensory neurons that are susceptible to auditory adaptation recapitulates these behavioral results. Our results imply that the neural circuits supporting tactile frequency perception also process auditory signals. This finding is consistent with the notion of supramodal operators performing canonical operations, like temporal frequency processing, regardless of input modality.NEW & NOTEWORTHY Auditory signals can influence the tactile perception of temporal frequency. Multiple neural mechanisms could account for the perceptual interactions between contemporaneous auditory and tactile signals. Using a crossmodal adaptation paradigm, we found that auditory adaptation causes frequency- and feature-specific improvements in tactile perception. This crossmodal transfer of aftereffects between audition and touch implies that tactile frequency perception relies on neural circuits that also process auditory frequency.

Feeling form: the neural basis of haptic shape perception.

The tactile perception of the shape of objects critically guides our ability to interact with them. In this review, we describe how shape information is processed as it ascends the somatosensory neuraxis of primates. At the somatosensory periphery, spatial form is represented in the spatial patterns of activation evoked across populations of mechanoreceptive afferents. In the cerebral cortex, neurons respond selectively to particular spatial features, like orientation and curvature. While feature selectivity of neurons in the earlier processing stages can be understood in terms of linear receptive field models, higher order somatosensory neurons exhibit nonlinear response properties that result in tuning for more complex geometrical features. In fact, tactile shape processing bears remarkable analogies to its visual counterpart and the two may rely on shared neural circuitry. Furthermore, one of the unique aspects of primate somatosensation is that it contains a deformable sensory sheet. Because the relative positions of cutaneous mechanoreceptors depend on the conformation of the hand, the haptic perception of three-dimensional objects requires the integration of cutaneous and proprioceptive signals, an integration that is observed throughout somatosensory cortex.

Dissecting neural circuits for multisensory integration and crossmodal processing.

We rely on rich and complex sensory information to perceive and understand our environment. Our multisensory experience of the world depends on the brain's remarkable ability to combine signals across sensory systems. Behavioural, neurophysiological and neuroimaging experiments have established principles of multisensory integration and candidate neural mechanisms. Here we review how targeted manipulation of neural activity using invasive and non-invasive neuromodulation techniques have advanced our understanding of multisensory processing. Neuromodulation studies have provided detailed characterizations of brain networks causally involved in multisensory integration. Despite substantial progress, important questions regarding multisensory networks remain unanswered. Critically, experimental approaches will need to be combined with theory in order to understand how distributed activity across multisensory networks collectively supports perception.

Motor system contributions to verbal and non-verbal working memory.

Working memory (WM) involves the ability to maintain and manipulate information held in mind. Neuroimaging studies have shown that secondary motor areas activate during WM for verbal content (e.g., words or letters), in the absence of primary motor area activation. This activation pattern may reflect an inner speech mechanism supporting online phonological rehearsal. Here, we examined the causal relationship between motor system activity and WM processing by using transcranial magnetic stimulation (TMS) to manipulate motor system activity during WM rehearsal. We tested WM performance for verbalizable (words and pseudowords) and non-verbalizable (Chinese characters) visual information. We predicted that disruption of motor circuits would specifically affect WM processing of verbalizable information. We found that TMS targeting motor cortex slowed response times (RTs) on verbal WM trials with high (pseudoword) vs. low (real word) phonological load. However, non-verbal WM trials were also significantly slowed with motor TMS. WM performance was unaffected by sham stimulation or TMS over visual cortex (VC). Self-reported use of motor strategy predicted the degree of motor stimulation disruption on WM performance. These results provide evidence of the motor system's contributions to verbal and non-verbal WM processing. We speculate that the motor system supports WM by creating motor traces consistent with the type of information being rehearsed during maintenance.

Static field influences on transcranial magnetic stimulation: considerations for TMS in the scanner environment.

BACKGROUND: Transcranial magnetic stimulation (TMS) can be combined with functional magnetic resonance imaging (fMRI) to simultaneously manipulate and monitor human cortical responses. Although tremendous efforts have been directed at characterizing the impact of TMS on image acquisition, the influence of the scanner's static field on the TMS coil has received limited attention. OBJECTIVE/HYPOTHESIS: The aim of this study was to characterize the influence of the scanner's static field on TMS. We hypothesized that spatial variations in the static field could account for TMS field variations in the scanner environment. METHODS: Using an MRI-compatible TMS coil, we estimated TMS field strengths based on TMS-induced voltage changes measured in a search coil. We compared peak field strengths obtained with the TMS coil positioned at different locations (B0 field vs fringe field) and orientations in the static field. We also measured the scanner's static field to derive a field map to account for TMS field variations. RESULTS: TMS field strength scaled depending on coil location and orientation with respect to the static field. Larger TMS field variations were observed in fringe field regions near the gantry as compared to regions inside the bore or further removed from the bore. The scanner's static field also exhibited the greatest spatial variations in fringe field regions near the gantry. CONCLUSIONS: The scanner's static field influences TMS fields and spatial variations in the static field correlate with TMS field variations. Coil orientation changes in the B0 field did not result in substantial TMS field variations. TMS field variations can be minimized by delivering TMS in the bore or outside of the 0-70 cm region from the bore entrance.

Feeling better: separate pathways for targeted enhancement of spatial and temporal touch.

People perceive spatial form and temporal frequency through touch. Although distinct somatosensory neurons represent spatial and temporal information, these neural populations are intermixed throughout the somatosensory system. Here, we show that spatial and temporal touch can be dissociated and separately enhanced via cortical pathways that are normally associated with vision and audition. In Experiments 1 and 2, we found that anodal transcranial direct current stimulation (tDCS) applied over visual cortex, but not auditory cortex, enhances tactile perception of spatial orientation. In Experiments 3 and 4, we found that anodal tDCS over auditory cortex, but not visual cortex, enhances tactile perception of temporal frequency. This double dissociation reveals separate cortical pathways that selectively support spatial and temporal channels. These results bolster the emerging view that sensory areas process multiple modalities and suggest that supramodal domains may be more fundamental to cortical organization.

Efficient and robust identification of cortical targets in concurrent TMS-fMRI experiments.

Transcranial magnetic stimulation (TMS) can be delivered during fMRI scans to evoke BOLD responses in distributed brain networks. While concurrent TMS-fMRI offers a potentially powerful tool for non-invasively investigating functional human neuroanatomy, the technique is currently limited by the lack of methods to rapidly and precisely localize targeted brain regions - a reliable procedure is necessary for validly relating stimulation targets to BOLD activation patterns, especially for cortical targets outside of motor and visual regions. Here we describe a convenient and practical method for visualizing coil position (in the scanner) and identifying the cortical location of TMS targets without requiring any calibration or any particular coil-mounting device. We quantified the precision and reliability of the target position estimates by testing the marker processing procedure on data from 9 scan sessions: Rigorous testing of the localization procedure revealed minimal variability in coil and target position estimates. We validated the marker processing procedure in concurrent TMS-fMRI experiments characterizing motor network connectivity. Together, these results indicate that our efficient method accurately and reliably identifies TMS targets in the MR scanner, which can be useful during scan sessions for optimizing coil placement and also for post-scan outlier identification. Notably, this method can be used generally to identify the position and orientation of MR-compatible hardware placed near the head in the MR scanner.

Representation of tactile curvature in macaque somatosensory area 2.

Tactile shape information is elaborated in a cortical hierarchy spanning primary (SI) and secondary somatosensory cortex (SII). Indeed, SI neurons in areas 3b and 1 encode simple contour features such as small oriented bars and edges, whereas higher order SII neurons represent large curved contour features such as angles and arcs. However, neural coding of these contour features has not been systematically characterized in area 2, the most caudal SI subdivision in the postcentral gyrus. In the present study, we analyzed area 2 neural responses to embossed oriented bars and curved contour fragments to establish whether curvature representations are generated in the postcentral gyrus. We found that many area 2 neurons (26 of 112) exhibit clear curvature tuning, preferring contours pointing in a particular direction. Fewer area 2 neurons (15 of 112) show preferences for oriented bars. Because area 2 response patterns closely resembled SII patterns, we also compared area 2 and SII response time courses to characterize the temporal dynamics of curvature synthesis in the somatosensory system. We found that curvature representations develop and peak concurrently in area 2 and SII. These results reveal that transitions from orientation tuning to curvature selectivity in the somatosensory cortical hierarchy occur within SI rather than between SI and SII.